Validation of Sepsis-3 using survival analysis and clinical evaluation of quick SOFA, SIRS, and burn-specific SIRS for sepsis in burn patients with suspected infection

Purpose Sepsis-3 is a life-threatening organ dysfunction caused by dysregulated host responses to infection; and defined using the Sepsis-3 criteria, introduced in 2016, however, the criteria need to be validated in specific clinical fields. We investigated mortality prediction and compared the diagnostic performance of quick Sequential Organ Failure Assessment (qSOFA), systemic inflammatory response syndrome (SIRS), and burn-specific SIRS (bSIRS) in burn patients. Methods This single-center retrospective cohort study examined burn patients in Seoul, Korea during January 2010–December 2020. Overall, 1,391 patients with suspected infection were divided into four sepsis groups using SOFA, qSOFA, SIRS, and burn-specific SIRS. Results Hazard ratios (HRs) of all unadjusted models were statistically significant; however, the HR (0.726, p = 0.0080.001) in the SIRS ≥2 group is below 1. In the adjusted model, HRs of the SOFA ≥2 (2.426, <0.001), qSOFA ≥2 (7.198, p<0.001), and SIRS ≥2 (0.575, p<0.001) groups were significant. The diagnostic performance of dichotomized qSOFA, SIRS, and bSIRS for sepsis was defined by the Sepsis-3 criteria. The mean onset day was 4.13±2.97 according to Sepsis-3. The sensitivity of SIRS (0.989, 95% confidence interval [CI]: 0.982–0.994) was higher than that of qSOFA (0.841, 95% CI: 0.819–0.861) and bSIRS (0.803, 95% CI: 0.779–0.825). Specificities of qSOFA (0.929, 95% CI: 0.876–0.964) and bSIRS (0.922, 95% CI: 0.868–0.959) were higher than those of SIRS (0.461, 95% CI: 0.381–0.543). Conclusion Sepsis-3 is a good alternative diagnostic tool because it reflects sepsis severity without delaying diagnosis. SIRS showed higher sensitivity than qSOFA and bSIRS and may therefore more adequately diagnose sepsis.


Introduction
Sepsis is a systemic response to infection and is one of the most common causes of organ dysfunction and mortality in burn patients and admission to intensive care units (ICUs) among critically ill patients [1,2]. In particular, the susceptibility for infection or sepsis increases in proportion to the total body surface area (TBSA) burned [3], and the systemic response caused by the burn itself is difficult to distinguish from the systemic inflammatory response syndrome (SIRS) developed during sepsis. Therefore, the diagnosis of sepsis remains challenging, and SIRS and conventional sepsis are not appropriate in burn patients [4,5]. Therefore, "the Consensus Conference to Define Sepsis and Infection in Burns" was held in Tucson on January 20, 2007, and it proposed alternative definitions of SIRS (bSIRS) and sepsis (i.e., burn sepsis [BS]) [6]. In 2016, Sepsis-3, which discards SIRS and severe sepsis, was redefined using the Sequential Organ Failure Assessment (SOFA) [7]. Many patients with SIRS have no infection, while some patients without SIRS have an infection; SIRS used to diagnose sepsis failed to detect organ dysfunction in one of eight infected patients with organ dysfunction [8,9]. The quick SOFA (qSOFA) has also been suggested as a bedside screening tool to detect organ dysfunction; however, its use was criticized given the delayed diagnosis that relies on established organ dysfunction, potentially decreasing the sensitivity [10][11][12]. Since the introduction of Sepsis-3, many studies have validated the new Sepsis-3 and qSOFA in several clinical fields such as emergency departments [13] and ICUs [14] in different medical conditions [15,16]. However, only few studies have assessed the validation of sepsis-3 in patients with burns. In this study, we investigated the clinical outcome by evaluating the ability of the tools in predicting morality for each sepsis category using survival analyses and assessing the diagnostic performance of SIRS, qSOFA, and bSIRS for sepsis defined by the Sepsis-3 criteria in burn patients with suspected infection.

Study design and population
In this single-center retrospective cohort study, we reviewed electronic medical records of patients with suspected infection from January 2010 to December 2020. The patients were aged over 18 years, were admitted to the Burn Intensive Care Unit (BICU) of our institution within 2 days after the burns, and had undergone acute fluid resuscitation during the first 2 days of admission. Overall, 2384 patients aged over 18 years were included, and we excluded 440 patients because they were burned 24 hours before admission, 185 owing to incomplete fluid resuscitation, and 368 owing to no-suspected infection. In total, 1,391 eligible patients were divided into four subgroups according to different criteria (Fig 1). All analyzed patients were primarily admitted as they had burns and a suspected infection during BICU admission. This study was approved by the Institutional Review Board of Hangang Sacred Heart Hospital (HG2017-063). The requirement for informed consent was waived owing to the retrospective nature of the study.

Data collection and missing values
The electrical medical records were anonymized and retrieved from the clinical database warehouse in Hangang Sacred Heart Hospital, Hallym University, Seoul, Korea. The database consists of structured data collected prospectively for all patients admitted to the BICU. Patient demographics such as age, sex, comorbidities, and variables for burn-specific scoring systems such as the Abbreviated Burn Severity Index (ABSI) [17], Hangang [18], and revised Baux (rBaux) [19] scores were calculated for each patient upon admission. For critically ill patients, additional scores, such as the Acute Physiology and Chronic Health Evaluation (APACHE) IV, were calculated [20]. The proportion of missing data at admission was <5% for all variables. These missing values reflected tests not being performed at the physicians' discretion under the assumption that they would return normal results; therefore, these missing values were replaced by normal values in the analyses. The scores of SOFA, qSOFA, SIRS, and bSIRS were calculated daily. Missing data regarding longitudinal variables were replaced with the values at the last observation. Only four longitudinal biomarkers (prothrombin time, lactate, total bilirubin, and bicarbonate) had more than 30% missing data. If the measured data were collected several times per day, the worst value of the day was included. The proportions and patterns of missing baseline and daily data are shown in S1 and S2 Figs in S1 File.

Outcomes and definitions
The primary outcome was 60-day in-ICU mortality. The secondary outcome was ICU length of stay (LOS). The gold standard for the diagnosis of sepsis (i.e., S3) was defined as an acute change in total SOFA scores of �2 points according to the Sepsis-3 criteria. Sepsis-3q (S3q) was defined as a qSOFA score �2, consisting of an altered mental state, respiratory rate �22 breaths/min, and systolic blood pressure �100 mmHg [7]. Sepsis-1 (S1) was defined as a SIRS score �2, consisting of a temperature >38˚C or <36˚C, heart rate >90 beats/min, respiratory rate >20 breaths/min, and white blood cell count >12,000 cells/mm 3 , <4,000/mm 3 , or >10% immature (band) forms. BS was defined as bSIRS �3, consisting of (1) a temperature >39˚C or <36.5˚C, (2) progressive tachycardia >110 bpm, (3) progressive tachypnea >25 bpm (not ventilated) or with minute ventilation >12 L/min, (4) thrombocytopenia <100,000 cells/μl, (5) hyperglycemia (in the absence of pre-existing diabetes mellitus) as indicated by an untreated plasma glucose >200 mg/dl, or by the presence of insulin resistance, and (6) inability to continue enteral feedings over 24 hours [6]. A suspected infection was defined as a situation where cultures to determine the infection source (i.e., blood, wounds, sputum, urine) were performed and antibiotics were administered before the results were received. A documented infection was defined as the presence of positive culture infection, pathologic tissue source, or clinical response to antimicrobials. Four sepsis criteria were assessed on a daily basis throughout the study period.

Statistical analysis
All continuous variables are expressed as means ± SD for normal distributions or median and interquartile ranges (IQR) for non-normal distributions. The independent t-test or Wilcoxon signed-rank test was used to determine differences between the two groups. Categorical variables were analyzed using the chi-squared test, and the frequencies are presented as percentages. The differences in the onset day for each sepsis category were compared using paired ttests, with the onset day of each category for Sepsis-3 as the reference. Accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for the diagnosis of sepsis were calculated. The hazard ratios (HRs) of in-ICU mortality for four dichotomized groups (i.e., S3, S3q, S1, and BS) were estimated using an extended Cox proportional hazards model to obtain longitudinal insight into how the individual components of the model affected the overall mortality of patients [21]. HR was also calculated after adjusting for known predictors of burn severity such as age, TBSA burned, and inhalation injury. Harrell's C-index was used to evaluate the prediction accuracy of the models. P <0.05 (two-sided) was considered to be statistically significant. Data management and statistical analysis were performed using R (A language and environment for statistical computing, Vienna, Austria, Version 4.1.0).

Study population
In this study, 1,391 burn patients with a suspected infection between January 2010 and December 2021 were included. All patients were divided into four groups according to the sepsis criteria. In total, 1,237 patients were diagnosed as having Sepsis-3 (S3), 1,307 patients as having Sepsis-1 (S1), 1,051 patients as having Sepsis-3q (S3q), and 1,005 patients as having BS (Fig 1  and Table 1).

Survival analysis according to the sepsis criteria
All survival curves showed statistically significant differences in the sepsis criteria between the groups using log-rank tests. All HRs in the unadjusted models were statistically significant; however, the HR (0.726, p = 0.008) in patients diagnosed using Sepsis-1 was lower than that in patients diagnosed using other sepsis criteria. In the adjusted model, the HRs of Sepsis-3

Detection days of each sepsis criteria
We analyzed the first detection day for each sepsis criteria to evaluate whether Sepsis-3 could diagnose sepsis early in the 909 patients who met all four sepsis criteria. The intersection of the four sepsis categories is presented in S4 Fig in S1 File. The mean onset day was the lowest with 4.13 ± 2.97 in Sepsis-3 and was statistically different with the onset in Sepsis-1 (4.23 ± 2.83), Sepsis-3q (6.26 ± 4.76), and BS (5.36 ± 4.88), as shown in S2

Diagnostic performance comparison of qSOFA, SIRS, and bSIRS for Sepsis-3
We evaluated the diagnostic performance of dichotomized qSOFA, SIRS, and bSIRS for sepsis defined by the Sepsis-3 criteria. The flow chart of the three divided groups is shown in S5 Fig  in S1 File.

Key findings
This large cohort study of burned patients with suspected infection from South Korea showed the clinical utility of the Sepsis-3 criteria to diagnose sepsis and distinguish the systemic

PLOS ONE
Validation of Sepsis-3, qSOFA, and SIRS in burn patients response induced by sepsis from that induced by burns themselves. The Sepsis-3 criteria can predict mortality even when adjusted for known burn predictors without delaying diagnosis. The Sepsis-1 criteria can diagnose sepsis similar to the Sepsis-3 criteria in the early days; however, it can predict the mortality reversely with an adjusted HR of 0.646. The qSOFA and bSIRS criteria showed high specificity and low sensitivity; therefore, limiting their utility as diagnostic tools. There is no need to use diagnostic criteria specific to burn patients in this cohort; thus, the Sepsis-3 criteria can be used to compare them with patients with other diseases associated with sepsis. Sepsis-3, defined by SOFA �2, can be useful as an alternative diagnostic tool.

Relationship to previous studies
We previously reported a somewhat different conclusion. In our previous study, the Sepsis-3 criteria were not superior to the other BS criteria in the present study [22]. This is because the performance of mortality prediction was evaluated cross-sectionally at the time of diagnosis. The evaluation of the cut-off value of SOFA as a Sepsis-3 criterion might be limited as a diagnostic tool. Burns and trauma are two well-known conditions in which SIRS develops without infections, implying that that SIRS may not immediately detect sepsis in this population. Therefore, the diagnosis of sepsis in burn patients is complicated by the inability to distinguish the systemic response induced by infection and the burn itself. Burn physicians ignore SIRS as it is considered to be a normal response. Due to this, there are cases in which patients with sepsis are missed, although the actual sepsis is progressing [23]. Therefore, the modified definition of SIRS is proposed by ABA and is determined by meeting at least three variables among the following: temperature, heart rate, respiration rate, platelet count, blood glucose, and intolerance of feeding [6]. Nevertheless, several studies have reported the best indicators of sepsis in thermally injured patients [24]. Yan et al. reported that Sepsis-3 showed higher sensitivity (85%) than BS (59%), defined by ABA, and that multiple organ dysfunction is more specific for sepsis than inflammation. However, their dataset did not collect all relevant biomarker data prospectively, and their study included a smaller cohort than our own [4]. In this study, we found that the SIRS is too sensitive with a low specificity to detect sepsis. SIRS could not stratify the severity of sepsis based on our results that the HR ratio is lower than 1. This showed a negative prediction for the group, in which sepsis was defined by SIRS �2. bSIRS has high specificity and lower sensitivity, which reflects the severity but is not an appropriate diagnostic tool. Additionally, it showed adjusted HR values of 0.966, indicating that it cannot stratify the risk of mortality and could be affected by burn factors but not sepsis itself. qSOFA showed similar sensitivity and specificity to bSIRS but not to bSIRS; its HR showed a better discriminative power for mortality prediction. The specificity of qSOFA was higher than that of SIRS in our study and was comparable to that reported in other studies [15,25].

Strengths and limitations
This study represents the largest series ever reported in the literature in this population.
Although this was a retrospective study, the data were prospectively collected and stored in an electronic data warehouse. Our study is the first evaluation of qSOFA in burn patients with suspected infection, and it showed that qSOFA had similar clinical sensitivity and specificity as bSIRS. It also used an extended Cox analysis to determine the predictive ability of daily-collected SIRS, SOFA, qSOFA, and bSIRS scores, as almost always, the infection and sepsis-like clinical symptoms occur in burn patients throughout ICU hospitalization. We also evaluated the onset days of each sepsis category in the same cohort to determine whether diagnosis was delayed. However, this study had several limitations. First, this was a single-center study that reflects the specific case-mix of our center, and we recognize that the results will depend on our population characteristics and their specific treatments. Second, qSOFA was introduced as a bedside screening tool, but it was evaluated as a diagnostic tool for sepsis in this study. Third, missing data were considered to be normal or previous values. Fourth, regarding the definition of a suspected infection, it is assumed that the culture was undertaken when the physician suspected an infection; however, it cannot be ascertained whether this was the actual time of suspicion or routine tests. Fifth, the primary outcome in this study is mortality, which cannot be used to definitively validate the ability to predict sepsis. However, given that it is very difficult to detect the systemic response associated with sepsis, numerous physicians simply refer to "septic" patients, rather than strictly defined "sepsis." Therefore, prediction of mortality, which is a crucial indicator in critical care patients, is an indirect way of evaluating the efficacy of a diagnostic tool for sepsis [14]. Furthermore, several studies have reported the validation of sepsis diagnostic tools using mortality as the primary outcome [15].

Conclusion
Sepsis defined by SOFA �2 can be a good alternative diagnostic tool because it reflects the severity of sepsis well using the survival analysis. SIRS showed higher sensitivity and lower specificity, while qSOFA and bSIRS showed higher specificity and lower sensitivity for the diagnosis of sepsis; therefore, they are not adequate for the diagnosis of sepsis.

PLOS ONE
Validation of Sepsis-3, qSOFA, and SIRS in burn patients